Image coding system

ABSTRACT

An image coding system determines an amount a data that is output by a coder of the system and also determines a target code produced amount based on the motion of an input image. The coder of the coding system encodes the input image. A record of a data amount of past coded data is used as the inference of the target code amount. Hence, proper coding can be performed in a sending side. Further, by converting the data amount into a value with a condition to be standard, accuracy of the inference may be improved. In addition, an indication of the power amount of the data to be coded is used as a factor for determining a target code produced amount to expedite a combination of a plurality of coding modes.

This application is a division of application Ser. No. 884,635filed May15, 1992, abandoned.

BACKGROUND OF THE INVENTION

i) Field of the Invention

The present invention relates generally to an image coding system and,more particularly, to an image coding system for carrying out a codingprocess based on motion in a temporally changing image.

ii) Description of the Prior Art

Pixel illuminance data of images is typically transmitted as a sequenceof digitally encoded frames. Each frame contains sufficient pixel datafor complete coverage of a video display. In the simplext case, a framecontains pixel data for all of the pixels in a display. For televisionapplications, frames are transmitted at a rate of thirty frames persecond. Given the large number of pixels in a typical display and giventhe rate at which the frames are transmitted (i.e., the frame rate),video transmission generally requires the transmittal of large amountsof data. Hence, image data is typically coded to compress the data sothat the data may be more efficiently transmitted. The amount ofcompression realized when coding is performed varies in accordance witha number of factors, including the coding method employed, the amount ofmovement in the image, the spatial resolution, and the temporalresolution.

The amount of data sent per frame affects the quality of the reproducedimage. For example, as shown in FIG. 1, when the amount of data includedper frame is increased (see the portion of the curve 2 where the framerate, expressed as I/B_(m), where B_(m) is a target information amount,is low), spatial resolution increases and, thus, the image quality(i.e., the signal to noise ratio (S/N)) of each frame is raised.However, the number of frames that can be transmitted per unit time isreduced (i.e., the frame rate decreases) so that temporal resolutiondrops. On the other hand, when the amount of data per frame decreases,the image quality also decreases, but the motion follow-up performanceincreases as shown by the curve 2 in FIG. 1.

Another approach to reducing the amount of data in a transmitted frameis to use a filter that removes a percentage of the image data. Forexample, suppose that the data is the illuminance difference betweencorresponding pixels in a present frame and a preceding frame. Theamount of data that is transmitted may be reduced by increasing athreshold value, which the illuminance differences must exceed to betransmitted (curve 6 in FIG. 2 shows an example wherein the filterthreshold is raised from T₁ to T₂). As a result, the transmittedinformation amount decreases from B₁ to B₂. The thresholds are switchedto raise the efficiency of the image transmission based on the image tobe transmitted.

The switching of thresholds to regulate information amount may beperformed by an image transmission controller such as disclosed inJapanese Patent Laid-open No. HEI 3-295392. The system of that patentincludes a receiving side where the image is reproduced from thetransmitted data. This receiving side operates by determining aproductive data amount for a frame. This data amount is then sent to thesending side, wherein coding processing is carried out according to thedata amount specified by the receiving side. FIG. 2 shows an example ofthe threshold values that must be selected to realize this data amountB_(m) for different images. In particular, for the image represented bycurve 8, threshold T₁ must be chosen in order to realize the specifieddata amount B_(M). In contrast, for the image of curve 4, a higherthreshold, T₂, must be chosen to realize the data amount B_(M).

In this system, the data for switching thresholds must be sent from thereceiving side to the sending side. Hence, in a system lacking a datatransmission means, switching of thresholds cannot be carried out.Further, in the case where the data amount is changed by only changingthe threshold value, as described above, a delicate image quality andmotion follow-up performance is difficult to achieve, and, thus, theefficiency of the image data transmission is poor.

SUMMARY OF THE INVENTION

The present invention provides a coding control apparatus that is ableto overcome the above-described difficulties of the prior art. Inaccordance with the present invention, an image system has a coder forcoding a sequence of image signals as dictated by a control signal thatsets coding parameters for the coder. The image coding system alsoincludes a controller for controlling the coder. The controller includesa motion detector for detecting an extent of motion between an imagesignal and at least one previously encoded image signal based on aninformation amount of the current image signal and an averageinformation amount of at least one previously encoded image signal. Theextent of motion detected by the motion detector is passed on to acoding update means. The coding update means generates a control signalthat sets the coding parameters based on current coding parameters andextent of motion detected by the motion detector. The coding updatemeans may be realized as an inference means for inferring the controlsignal using a control rule based on the extent of motion detected bythe motion detector and current coding parameters. Likewise, the codingupdate means may alternatively be realized as a coding parameter setmeans that generates a control signal so that the coding produces atarget information amount.

In accordance with one embodiment, the coding control apparatus codes aframe of an image data according to the motion between image data offrame and image data of at least one previous frame. In this embodiment,the coding control apparatus includes a first memory for storing motioncharacteristics of the image data of the previous frames. The codingcontrol apparatus further includes a second memory for storing a codingparameter to control coding of the image data of the present frame sothat coded image data has a produced information amount. A detectordetects motion over a predetermined sequence of frames from the pastmotion characteristics held in the first memory. The coding parametersare updated via an inference means, such as that described above.

In another alternative embodiment, the image coding apparatus includes aconverter for converting the produced information amount of the codeddata into a conversion coded data amount, while maintaining a fixedrelationship between a predetermined quantized condition and theproduced information amount. In this alternative embodiment, a quantizedupdate means calculates the quantize condition and updates the quantizecondition in the coding means according to the motion characteristicsobtained by the detector.

In a third embodiment, the apparatus includes an image processor forreceiving input image data and processing the input image data so thatit is compatible with the coder. The coder codes the input image dataunder a predetermined quantize condition and a plurality of differentcoding modes. Each mode codes the data in a different fashion. In thisthird embodiment, the apparatus includes a power calculater forcalculating estimated output power as a sum of a size of the data thatis output by the image processor. This third embodiment further includesa coded parameter set means like that described above.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features and advantages of the presentinvention will more fully appear from the following description of thepreferred embodiments with reference to the accompanying drawings, inwhich:

FIG. 1 is a graph showing the relationship between frame rate and imagequality in a conventional coding controller;

FIG. 2 is a graph showing the relationship between threshold value and aproduced information amount in a conventional coding controller;

FIG. 3 is a block diagram of a coding processor in a sending sideaccording to a first embodiment of the present invention;

FIG. 4 is a block diagram of a coding controller according to a firstembodiment of the present invention;

FIG. 5 is a more detailed block diagram of the coder 14 of FIG. 3;

FIG. 6 is a more detailed block diagram of the local decoder 12 of FIG.3;

FIG. 7 is a more detailed block diagram of the motion detector 153 ofFIG. 4.

FIGS. 8A-8G are schematic views explaining a coding control operation byfuzzy inference in a fuzzy inference unit according to the firstembodiment of the present invention;

FIG. 9 is a graph showing a change in the amount of produced informationfor long, intermediate and short periods of motion with respect toelapsed time according to the first embodiment of the present invention;

FIGS. 10A-10H are schematic views showing one example of a control ruleused for the fuzzy inference in the fuzzy inference unit according tothe first embodiment of the present invention;

FIG. 11 is a conceptual view of the operation of the coding controlleraccording to the first embodiment of the present invention;

FIG. 12 is a block diagram of a coding controller according to a secondembodiment of the present invention;

FIG. 13 is a schematic view showing one example of a quantizecharacteristic value control mode by the coding controller according tothe second embodiment of the present invention;

FIG. 14 is a graph of the relationship between a quantize characteristicvalue and a produced information amount according to the secondembodiment of the present invention;

FIG. 15 is a block diagram of a coding controller according to a thirdembodiment of the present invention;

FIG. 16 is a more detailed block diagram of the parameter renew unit 140of FIG. 15.

FIG. 17 is a block diagram of a coding processor in a sending sideaccording to the third embodiment of the present invention;

FIG. 18 is a schematic view showing a period of frames to be coded bydifferent coding modes according to the third embodiment of the presentinvention;

FIG. 19 is a graphical representation showing the relationship between aproduced information amount target value and a coding mode according tothe third embodiment of the present invention;

FIG. 20 is a graphical representation showing the relationship betweenthe produced information amount target value and an estimated errorpower and period according to the third embodiment of the presentinvention;

FIG. 21 is a graphical representation showing the relationship betweenthe quantize characteristic value and the produced information amounttarget value and the estimated error power according to the thirdembodiment of the present invention; and

FIG. 22 is a flowchart of a method in accordance with the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will now be described in connection with itspreferred embodiments with reference to the attached drawings, whereinlike reference characters designate like or corresponding partsthroughout different views, and, thus, the repeated description thereofis omitted for brevity.

FIG. 3 shows a coding processor in a sending side of a system fortransmitting image data, in accordance with the first embodiment of thepresent invention. In this embodiment, an image processor 10 receivesinput image data 11 and decoded data 13 of previously received inputimage data that is sent from a sending side local decoder 12. The localdecoder 12 locally decodes the coded data 17 of previously receivedinput image data that is output by a coder 14. The image processor 10processes the input image data 11 and decoded data 13 to output either avalue that is indicative of the difference between the input andprevious data or input image data 11.

The data output from the image processor 10 is coded by the coder 14 inaccordance with coding parameters 16 that are sent from the codingcontroller 15. The coding parameters 16 will be described in more detailbelow.

FIG. 5 provides a more detailed depiction of the coder 14. The coderincludes a transform circuit 200 for performing a transformation such asa discrete cosine transfer (DCT) on the input image data 11. Thetransformed data is passed to a quantizing circuit 202 where the data isquantized to produce coded data. Quantization is controlled by aquantize characteristic value that is passed to the coder 14 as part ofthe coding parameters 16. The quantize characteristic value controls theamount of information that is coded by controlling the quantization ofthe data.

The coder 14 outputs the coded data 17 to the local decoder 12. FIG. 6provides a more detailed view of the local decoder 12. The local decoderreceives the coded data 17 and forwards the data to a requantizer 206,wherein the data is requantized. The requantized data is then passed toan inverse discrete cosine transform (IDCT) circuit 208 that performs aninverse discrete cosine transform on the requantized data. Thetransformed data is then added with data from the previous frame byadder 210 to produce the decoded image data. The image data of theprevious frame is retrieved from a frame memory 212.

The coded data is also sent to the transmission buffer 18 whichtemporarily stores the coded data 17. The transmission buffer 18determines and outputs the produced information amount 19 to the codingcontroller 15. The coded data is then transmitted down a transmissionline 5.

The above-described system operates as follows. For illustrativepurposes, suppose that the input image data is processed in pixel blockunits (hereinafter referred to in short as blocks) that are composed ofa plurality of pixels (such as 8 pixels×8 pixels). When the imageprocessor 10 is operating in an intra-frame estimation mode, the imageprocessor 10 receives the input image data 11 and outputs a block of theinput image data to the coder 14. In contrast, when the image processor10 is operating in an inter-frame estimation mode, the image processoroutputs a difference value between the block of the input image 11 andthe block of the estimated image 13 (i.e., the corresponding block ofthe proceeding frame). The image block data that is output from theimage processor 10 is then coded by the coder 14. The coder 14 encodesthe image block data so as to produce a coded output having the targetproduced information amount specified by the coding parameters 16. Thecoded data 17 is output from the coder 14 to the transmission line 5 viathe transmission buffer 18. The coded data 17 is also output to thelocal decoder 12. In the transmission buffer 18, the information amount19 in the unit that was processed (such as a block unit, frame unit orthe like) is output to the coding controller 15.

Before discussing the operation of the coding controller 15, it ishelpful to examine the components of the controller in more detail. FIG.4, shows the coding controller 15 in accordance with a first embodimentof the present invention. The controller 15 includes a memory 151 forstoring present motion characteristics, and the past motioncharacteristics of the long, intermediate and short periods. The pastmotion characteristics of the long, intermediate and short periods willbe described in more detail below. A motion characteristics detector 153receives motion characteristic information 152 read out of the memory151 and a present produced information amount 19 (corresponding to acode amount) per unit and detects and updates the present motioncharacteristics and the past motion characteristics of the long,intermediate and short periods.

FIG. 7 provides a more detailed depiction of the motion detector 153.The motion detector 153 includes a decoder 214 that decodes aquantization characteristic value that is included in the motioncharacteristic information sent from memory 152. The quantizationcharacteristic value is decoded to select a conversion constant entry ina conversion constant memory 216. This memory 216 holds a number ofconversion constants that are used in calculating updated motioninformation. Specifically, the selected conversion factor is multipliedby the present produced information amount by multiplier 218 to generatethe update motion characteristic information 154. The conversion factoris necessary because the information amount alone does not indicate theextent of motion in the coded image; rather, one must know thequantization characteristic value that was employed to produce theinformation amount. For instance, a large quantization characteristicvalue will produce a smaller information amount, even if there issubstantial motion in the coded image. Hence, the conversion factorprovides a means for accounting for what quantization characteristicvalue was used.

A fuzzy inference unit 155 (FIG. 4) receives updated motioncharacteristic information 154 that is output from the motioncharacteristics detector 153 and also receives coding parameters 157that are read out of a memory 156. The fuzzy inference unit 155 draws aninference therefrom as hereinafter described in detail to produce outputinferred coding parameters 16 that represent inferred coded amounts andinferred coded object ranges. The memory 156 stores records of theinferred coding parameters 16 output by the fuzzy inference unit 155 andadditionally stores the past coding parameters.

The present motion characteristics (See FIG. 9) are expressed as thedifference between the present produced information amount and thetarget information amount. The past motion characteristics of the longperiod are equal to the difference between the average value of the pastproduced information amounts during the long period and the targetinformation amount. The past motion characteristics of the intermediateperiod are equal to the difference between the average value of the pastproduced information amounts during the intermediate period and thetarget information amount. Lastly, the short period of past motioncharacteristics is expressed as the temporal change (time differentialof change or inclination) of the past produced information amounts ofthe short period. The coding parameters include the presentvalid/invalid discrimination threshold value, the quantizecharacteristic value, and the temporal change (time differential ofchange or inclination) during the short period of past valid/invaliddiscrimination threshold values and the quantize characteristic values.

The coding controller 15 operates as follows. The produced informationamount 19 and the long, past motion characteristics information 152 ofthe long, intermediate and short periods are input to the detector 153.The detector 153 updates the past motion characteristics of the long,intermediate and short periods on the basis of the newly input producedinformation amount and returns the updated motion characteristicinformation to the memory 151 wherein the updated information is stored.The detector 153 also outputs the difference between the target value atthe present time and the renewed past motion characteristic information154 of the long, intermediate and short periods to the fuzzy inferenceunit 155. In the fuzzy inference unit 155, the fuzzy inference of thecoding parameters is carried out for every coding unit (such as a blockunit, a frame unit, or the like) using the motion characteristicinformation 154 fed from the detector 153 and the coding parameters 157read out of the memory 156. Further, the inferred coding parameters 16are output to both the coder 14 and the memory 156. In the memory 156,the stored content is renewed.

The fuzzy inference operation of the coding parameter will now bedescribed. In the fuzzy inference unit 155, an inference is carried outby applying a control rule using a membership function that expresses anindistinct outlined object as a function. An output is inferreddepending on an indication of how much an input value adapts to acontrol rule. This indication is known as an adaptability. FIGS. 8A-8Gprovide an illustration of a coding control method using fuzzyinference. FIGS. 8A-8C illustrate a method for inferring an output by arule 1 such that if x₁ belongs to a fuzzy set of A₁ (see the triangulararea occupied by the membership function of A₁ in FIG. 8A), and x₂belongs to set AT₁ (see FIG. 8B), then, as shown in FIG. 8C, y belongsto set R₁ (if x₁ =A₁, x₂ =AT₁, then y=R₁). FIGS. 8D-8F illustrate amethod of inferring an output by a rule 2 such that if x₁ belongs to setA₂ (see FIG. 8D) and x₂ belongs to set AT₂ (see FIG. 8F), then, as shownin FIG. 8F, y belongs to set R₂ (if x₁ =A₂, x₂ =AT₂, then y=R₂). In therules shown in FIGS. 8A-8F, the if-clauses of rule 1 and rule 2 arereferred to as "antecedent parts", and the then-clauses of the rules arereferred to as "consequent parts".

Each rule expresses knowledge of a conventional control method in theform of sets. For example, rule 1 shows that, if a present producedinformation amount (x₁) is moderately greater than a target value (i.e.,x₁ lies in the set A₁) and a time variation of the produced informationamount (x₂) is moderately greater than the time variation target value(i.e., x₂ belongs to the set AT₁), a difference value (y) of a quantizecharacteristic value is determined to be a moderate value (R₁ ). Rule 2shows that, if the present produced information amount (x₁) is very muchgreater than the target value (in set A₂) and the time variation of theproduced information amount (x₂) is very much greater than the targetvalue (in set AT₂), the difference value (y) of the quantizecharacteristic value is determined to be a large value (R₂). If theserules are proper on the basis of past experiences, when information tobe put between rules 1 and 2 is input, a relatively correct inferencecan be readily carried out by using rules 1 and 2.

That is, when a value "a" for produced information amount A, whichbelongs to A₁ (FIG. 8A) and A₂ (FIG. 8D) and a value "b" of AT, whichbelongs to AT₁ (FIG. 8B) and AT₂ (FIG. 8E) are input to the fuzzyinference unit 155, adaptability of rule 1 and rule 2 can be obtained.Relating to the adaptability of rule 1, the lower value of theadaptability of A₁ and "a" and the adaptability of AT₁ and "b" isadopted. As an inference result of rule 1, a part R₁ ' of theadaptability, indicated by cross-hatching in FIG. 8C, of a set R₁ isobtained. In case of rule 2, the adaptability is obtained in the samemanner as rule 1 ("a" in rule 2 is adopted), and as an inference resultof rule 2, a part R₂ ' of the adaptability, indicated by thecross-hatching in FIG. 8F, of a set R₂ is obtained. The last inferenceresult of the input (a, b) is obtained by the sum set of the inferenceresults R₁ ' and R₂ ' obtained in the two rules, and its center ofgravity is indicated by R', which is shown in FIG. 8G. This center R' ofgravity shows that the difference value of the quantize characteristicvalue is somewhat greater than a median. Hence, an inference is obtainedsuch that, when the present produced information amount is slightlygreater than the median so that the present produced information amountis slightly larger than the target value and the time variation of theproduced information amount is fairly greater than the median, thedifference value of the quantize characteristic value is increasedslightly more than the median.

Although the basic approach used in obtaining the fuzzy inference hasbeen described above, in order to bring the inference result close tothe optimum value, it is convenient to provide a number of control ruleswith detailed conditions. When the dynamic image is coded, since theproduced information amount is sometimes rapidly increased by a scenechange or the like, it is necessary to seize the temporal motioncharacteristics for the long, intermediate and short periods relative tothe estimation point, as shown in FIG. 9. Further, since the imagequality and the motion follow-up performance are controlled by thecoding parameters, it is necessary to grasp the past records togetherwith the motion characteristics.

Hence, the antecedent part of the control rule uses the present and pastcoding parameters 157 (FIG. 4), which have been read out of the memory156, the present motion characteristics and the past motioncharacteristics 154 of the long, intermediate and short periods, whichhave been output from the detector 153.

FIGS. 10A-10H illustrate a membership function of a control ruleincluding antecedent and consequent parts. The antecedent part of thecontrol rule include a difference A between a produced informationamount and a target amount that is somewhat positive as shown in FIG.10A. In contrast, as shown in FIG. 10B, there is almost no change of theproduced information amount over the long period (see AT(B)), and, asshown in FIG. 10C, there is almost no change of the produced informationamount over the intermediate period (see AT(C)). FIG. 10D illustratesthat the change of the produced information amount over the short periodis somewhat positive (see AT(D)). There is almost no change of avalid/invalid discrimination threshold value (see FIG. 10E), and thereis almost no change RT of a difference value of a quantizecharacteristic value (see FIG. 10F). The consequent part of the controlrule includes a valid/invalid discrimination value Y that is somewhatpositive so as to moderately raise the rate of data that is deemedinvalid (see FIG. 10G), and also includes a quantize characteristicvalue R that is somewhat positive to reduce the code amount (see FIG.10H).

In practice, for both the antecedent and consequent parts, seven stages(such as a largely negative stage, a moderately negative stage, aslightly negative stage, an almost zero stage, a slightly positivestage, a moderately positive stage and a largely positive stage) ofmembership functions are prepared, and all possible combinations ofcontrol rules are made. Then, in the fuzzy inference unit 155, accordingto these combinations of control rules, a fuzzy inference is carried outfrom the present motion characteristics 154 sent from the detector 153and the coding parameters 157 read out of the memory 156, to obtain thesum set of the inference results of all of the control rules. The centerof gravity of the sum set of inference results, the valid/invaliddiscrimination threshold value, and the quantize characteristic valueare inferred and output to the coder 14.

In FIG. 11, there is shown a conceptual view of the operation of acoding controller in accordance with the first embodiment of the presentinvention. FIG. 8 provides a plot of variation amounts ΔY and ΔR of avalid/invalid discrimination threshold value Y and a quantizecharacteristic value R, respectively (see the vertical axis) and aproduced information amount A (see horizontal axis). As shown in FIG.11, when the produced information amount A is separated from the targetvalue A_(m), the variation amounts ΔY and ΔR are updated. Accordingly,the information amount that is output from the coder 14 may be close tothe target value A_(M). In FIG. 11, when the produced information amountA exceeds a certain value Amax, a time lapse control routine isperformed to receive a number of frames that are transmitted per unittime in order to maintain the image quality. Further, as describedabove, since the valid/invalid discrimination threshold value Y and thequantize characteristic value R cannot be inferred from only theproduced information amount, FIG. 11 lacks the accuracy of the formalexpression of the rate but, nevertheless, helps to illustrate theconcept. That is, by changing both the valid/invalid discriminationthreshold value Y and the quantize characteristic value R to negative inimages, the produced information amount A is made larger than the targetvalue A_(m). By changing both the valid/invalid discrimination thresholdvalue Y and the quantize characteristic value R to positive, theproduced information amount A is made smaller than the target valueA_(m). Of course, depending on the accuracy of control, a case where thecodes of the valid/invalid discrimination threshold value Y and thequantize characteristic value R are different from each other may beproperly handled.

In this embodiment, although the "min" operation which selects thesmaller-one of the two adaptable values, is carried out to obtain theadaptable value in the fuzzy inference unit 155 (FIG. 4), an algebraicproduct of the two adaptable values within the same rule (i.e., anintegration w_(1a) ×w_(1b) of an adaptable value w_(1a) of "a" againstA₁ and an adaptable value w_(1b) of "b" against AT₁ in FIGS. 8A-8G) maybe used. Other operation methods for carrying out a proper inference ineach rule may also be used. The present invention is not intended to berestricted to the above-described inference operation methods.

In this example although the sum set of the adaptable values of therules is calculated to obtain the center of gravity in order todetermine the last adaptable value (the inference value of the codingparameter), an addition of the adaptable values of the rules iscalculated to determine the center of gravity as the last adaptablevalue. The last adaptable value can be obtained by an operationreflecting the adaptable values of the rules. The present invention isnot restricted to these methods.

In this illustrative embodiment, the antecedent part of the control ruleis controlled by the six conditional sentences (such as the differencebetween the present produced information amount and the target value,the past motion characteristics of the long, intermediate and shortperiods, the valid/invalid discrimination threshold value). However,depending on the details of the control of the image quality and themotion, a reduction in the number of conditional sentences of theantecedent part, a reduction in the number of parameters that theconsequent part infers, an increase of a temporal change of a thresholdvalue of motion compensation into the conditional sentences of theantecedent part, and an addition of a difference value between presentand preceding threshold values of the motion compensation into theparameters that the consequent part infers may be performed. Thus, notonly the number but also the content of the conditional sentences may bevaried. Since the control rules may be described according to thecontrol accuracy, the present invention is not restricted by the numberand content of the conditional sentences of the antecedent part nor isit restricted by the number and content of the coding parameters thatthe consequent part infers.

In this illustrative embodiment, although the coding parameters used inthe antecedent part and the coding parameters used in the consequentpart are the same, the number and kinds of the coding parameters used inthe antecedent and consequent parts may be made different. The presentinvention is not restricted by the modifications. Furthermore, althoughthe inference of the coding parameter per block unit is carried out inthis illustrative embodiment, the inference can be executed per pixelunit, or per block group unit. Still further, the present invention isnot restricted to a particular type of processing unit. In the systemdescribed above, the above-described processes can be implemented inhardware or software, and the present invention is not restricted by anymanner of composition.

FIG. 12 shows a coding controller 15 according to a second embodiment ofthe present invention. In this second embodiment, a memory 21 storespresent and past motion characteristics, and a motion characteristicsdetector 23 receives various motion characteristic information 22 thathas been read out of the memory 21. The characteristics detector 23 alsoreceives the present produced information amount 19 for each unit thatis processed and updates the present and past motion characteristics. Acoding parameter renew unit 25 receives various renewed motioncharacteristic information 24 output from the motion characteristicsdetector 23 and various coding parameters 27 read out of a memory 26.The coding parameter renew unit 25 updates the coding parameters tooutput updated coding parameters 16. The memory 26 stores records of theupdated coding parameters 16 output by the coding parameters renew unit25 and the past coding parameters. The coding parameters 27 read out ofthe memory 26 are sent to the detector 23. The same coding processorthat is used in the first embodiment of FIG. 4 may be employed in thesecond embodiment of the present invention.

As shown in FIG. 12, the produced information amount 19, the past motioncharacteristics 22 that has been read out of the memory 21 and thevarious coding parameters 27 that have been read out of the memory 26are input to the detector 23. The detector 23 updates the past motioncharacteristics according to the produced information amount 19 andfeeds back the updated motion characteristics to the memory 21. Further,the detector 23 obtains a conversion information amount H as the motioncharacteristics from a quantize characteristic value Q and also obtainsthe produced information 19 of the coding parameter 27 from the memory26 according to the curves shown in FIG. 14. The detector 23 outputs thepresent and past motion characteristics 24 to the coding parameter renewunit 25. In the coding parameter renew unit 25, ΔQ is obtained from themotion characteristics 24 (the conversion produced information amountH), and ΔQ is added to the quantize characteristic value Q of the codingparameter 27 to update the value Q. The updated coding parameter 16 issent to the memory 26 and the coder 14 of the coding processor shown inFIG. 4.

FIG. 14 shows the relationship between the quantize characteristic valueQ and the produced information amount at a certain standard temporalresolution (i.e., frame rate). As shown in FIG. 14, the lower curverepresents the characteristics for coding the input images with lessmotion, and the upper curve represents the characteristics for codingthe input images with more motion. The greater the motion of the inputimages, the larger is the produced information amount. Hence, for agiven produced information amount A₁, as the motion of the input imagebecomes large, the quantize characteristic value Q becomes large. Thequantize characteristic value Q and the temporal resolution (the framerate) are selected to obtain a produced information amount H. Hence, thedegree of motion of the image may be accommodated by examining theconversion produced information amount.

The conversion produced information amount H is obtained as follows. InFIG. 14, for example, by using the standard characteristic curvedesignated X₁, when the actual quantize characteristic value is Q₂ andthe produced information amount is A₁, the point X₁ ' corresponding tothe value Q₂ is located on the curve, and the produced informationamount A₂ corresponding to the point X₁ ' becomes the conversionproduced information amount H. Such a conversion can be achieved bypreparing maps that are provided with conversion factors for thestandard characteristics, corresponding to A and Q values.

FIG. 13 illustrates a control example for renewing the quantizecharacteristics value Q depending on the motion amount or the conversionproduced information amount H. When the conversion produced informationamount is larger than the target value H_(m), the motion amount isconsidered to be large, and the present quantize characteristic value Qis enlarged (as shown by the increasing plot in FIG. 13) compared to thepreceding value Q₋₁ in order to reduce the produced information amount.In turn, when the conversion produced information amount is smaller thanthe target value H_(m), the motion amount is considered to be small, andthe present quantize characteristic value Q is diminished (as shown bythe decreasing plot in FIG. 13) relative to the preceding value Q₋₁ inorder to increase the produced information amount. Accordingly, thecontrol is carried out as described above so that the spatial resolution(coded image quality) may be close to the predetermined (uniform) value.When the conversion produced information amount H is more than a certainvalue or comes within a hatched area, a time lapse control method isperformed (i.e., a number of frames to be transmitted per unit time isreduced) in order to maintain the image quality above a certain level.

In this embodiment, although the renew value of the quantizecharacteristic value is determined from one conversion producedinformation amount H, the renew value of the quantize characteristicvalue may be decided from a plurality of conversion produced informationamounts, such as the present and past conversion produced informationamounts. Further, although the control of the coding parameter isdescribed as being carried out per block unit, it may be performed perpixel unit, or per block group unit including a plurality of blocks.Also, depending on the input image, the coding parameter control may beconducted only against a local block, or against selected frames or thelike, to perform the renewal of the parameter.

Still further, although a quantize characteristic value is used in theillustrative second embodiment, as the coding parameter relating to thespatial resolution, other coding parameters of the spatial resolutionmay be used, and a combination of such parameters may also be used.

FIG. 15 shows a coding controller 15 according to a third embodiment ofthe present invention. As shown in FIG. 15, an estimated error powerdiscriminator 110 receives the estimated error power 20 of an inputimage and an estimated image and discriminates the intensity or size ofthe estimated error power. A memory 120 stores the produced informationamount 19. A coding mode discriminator 130 discriminates coding modes tooutput the coding mode 21. A coding parameter renew unit 140 determinesa produced information amount target value B_(m) using the discriminatedintensity or size of the estimated error power that has been sent fromthe estimated error power discriminator 110. In determining B_(m), therenew unit 140 also uses the produced information amount that has beenread out of the memory 120, the coding mode that has been sent from thecoding mode discriminator 130 and a period T that has been determinedinternally. The renew unit then outputs the coding parameter 16 thatcaptures B_(m).

FIG. 16 provides a more detailed view of the parameter renew unit 140.The parameter renew unit 140 includes a selector 220 for selecting amongparameter tables such as 222a, 222b and 222c. The selector 220 uses theinputs of the encoding mode, the estimated error power and the producedinformation amount to select a table and entry in the table. Theselected entry is then output as the coding parameter 16.

FIG. 17 shows a coding processor in a sending side in accordance withthe third embodiment of the present invention. In this third embodiment,an image processor 10 receives the input image data 11 and the decodeddata 13 output from a sending side local decoder 12 through a localdecode image memory 22 and processes the received data. Depending on thecoding mode 21 that is output from the coding controller 15, theprocessor 10 processes the input and either outputs a difference valuebetween the input image data 11 and the decoded data 13 or outputs theinput image data 11. The image processor 10 also outputs the estimatederror power 20 to the coding controller 15. The local decoder 12 locallydecodes coded data 17 sent from a coder 14 in the sending side andoutputs the locally decoded data 13. The coder 14 codes the data outputfrom the image processor 10 in accordance with the coding parameters 16sent from the coding controller 15, which has been described above, tooutput the coded data 17. A transmission buffer 18 temporarily storesthe coded data 17 sent from the coder 14. The buffer 18 outputs theproduced information amount 19 to the coding controller 15 and alsooutputs the coded data 17 to a transmission line 5. The memory 22 storesthe image data of the locally decoded frames.

The operation of the system described above in connection with FIGS. 15and 17 will now be described in detail. First, the operation of asending side coding unit is described. FIG. 18 schematically shows aperiod T of frames to be coded by different coding modes. These are,mode A: a frame is coded by using only data within the current frame;mode B: a frame is estimated in one temporal direction; and mode C: aframe is estimated to both temporal directions. In the image processor10, in accordance with the coding mode 21 output from the codingcontroller 15 to the coder 14, the block (including 8×8 pixels of theinput image 11) is output in an intra-frame estimation mode denoted asmode A. On the other hand, difference data indicating the differencebetween the block of the input image 11 and the block of the estimatedimage 13 (the frame A or B shown in FIG. 18) that is output from thememory 22 is output when the coding moding an inter-frame one directionestimation mode denoted as mode B. Lastly, difference data indicatingthe difference between the block of the preceding or following framewith reference to the present frame to be coded are output when thecoding mode is in an inter-frame dual direction estimation mode denotesas mode C. For example, in FIG. 18, when the frame #2 is coded in modeC, the difference data between an arithmetic mean of frames #1 and #4and the input frame #2 is output to the coder 14.

In the coder 14, in accordance with the coding parameter 16 sent fromthe coding controller 15, the image block data input from the imageprocessor 10 are coded to output the coded data 17 to not only thetransmission line 5 via the transmission buffer 18 but also to outputthe coded data to the sending side local decoder 12. In the sending sidelocal decoder 12, the coded data 17 are locally decoded, and the decodeddata 13 are sent to the local decoded image memory 22 for storagetherein. In the transmission buffer 18, the information amount 19produced per unit that is processed (block unit in this embodiment) isoutput to the coding controller 15.

The operation of the coding controller 15 shown in FIG. 15 will now bedescribed. The produced information amount 19 output from thetransmission buffer 18 is sent to the memory 120 and is stored therein.On the other hand, the image processor 10 calculates the estimated errorpower P 20 of the data output depending on the chosen mode. The imageprocessor 10 sends the estimated error power P 20 to the estimated errorpower discriminator 110. The estimated error power P is the valuerepresenting the total size of the output data. The estimated errorpower P is the sum of the absolute value of the data of each pixel.Hence, in mode B or C, the estimated error power P becomes the sum ofthe absolute value of the difference data. In the estimated error powerdiscriminator 110, the size of the estimated error power P (thedifferential absolute value sum) 20 is discriminated, and the result isoutput to the coding parameter renew unit 140. In the coding modediscriminator 130, depending on the number of the frame to be coded, thecoding mode 21 is selected and output to both the image processor 10 andthe coding parameter renew unit 140. In the coding parameter renew unit140, as shown in FIG. 19, a base A_(m) of the produced informationamount target value that corresponds to the coding mode N is determined.As shown in FIG. 20, when it is found, from the discrimination result ofthe estimated error power discriminator 110, that the estimated errorpower 20 is large, a positive offset is added to the base A_(m). Thedegree of largeness of the estimated power determines the extent ofpositive offset. In turn, when the estimated error power 20 is small, anegative offset is added to the base A_(m). The degree of smallness ofthe estimated power determines the extent of negative offset. As aresult, the produced information amount target value A_(m) (hereinafterreferred to as A_(m) in short), to be used for estimating the codingparameter, is renewed. In this embodiment, the optimum determination ofthe target value for each frame can be carried out for an estimatederror power P.

FIG. 19 shows relationship between the coding method and the producedinformation mount H. In FIG. 19, A_(max) refers to a maximum value,A_(M) refers to a target value, A_(L) refers to a lower limit, andA_(min) refers to a minimum value of the produced information mount inmode A. The coding method N may also operate in modes B and C. Thesubscripts used for produced information amounts in these modescorrespond to the subscripts used for mode A.

In mode A, the produced information amount varies from A_(min) toA_(max), Analogously, in modes B and C, the produced information mountsvaries from B_(min) to B_(max) and C_(min) to C_(max), rspectively.Therefore, it is impossible to know the mode in which the coding processis being executed in when the produced information mount is smaller thanC_(max). For example, when operating in mode A on a monocolor image, theDCT conversion and quantization produces only DC components or low levelAC components. As such, only a small amount of information is produced.In modes B and C, when two frames have few images containing differentpixels and the images undergo DCT conversion and quantization, only DCcomponents or low level AC components are generated. In such instances,a small amount of produced information will be obtained.

The coding parameters are renewed depending upon the producedinformation amount regardless of the mode of operation, and thus, thequality of the decoded image depends upon the mode of operation.Specifically, if the produced information amount and the quantizecharacteristic value are the same for modes A, B and C, the quality forprecision of the decoded image for mode A will typically be inferior tothe quality of the decoded images for modes B and C. Therefore, thedecoded image produced in modes B or C will not necessarily satisfy thetarget quality for precision. This failure in quality stems primarilyfrom the decoded image obtained in mode A serving as a reference that isbelow the target quality. As such, more data concerning the differencebetween images is produced unnecessarily.

The target values of the produced information amounts are independentlydetermined for modes A, B and C so as to obtain a stable image quality.The produced information amounts are independently determined becauseinformation amounts tend to vary with coding mode. When the producedinformation amount does not meet the target value, the coding parameteris renewed. For example, if the produced information amount in aprevious mode (i.e., mode A) is larger than A_(M) in mode A, thequantize characteristic value Q is made larger than "1" (supposing thatthe previous value of Q was "3" and the present value of Q is "4").Otherwise, if the produced information amount in mode A is smaller thanA_(L) in mode A, the quantize characteristic value Q is made smallerthan the previous value Q.

If the previous produced information amount is too old, the presentframe will be temporarily coded by a preset quantize characteristicvalue. The data that is coded using this preset quantized characteristicvalue will be used without any modification according to the producedinformation amount at that moment. Otherwise, the coding will be carriedout again by renewing the value of Q. In addition, it is possible torenew the coding parameter according to the previous producedinformation amount and the produced information amount that istemporarily used to code the present frame.

In the above described example, the coding parameter is described as thequantize characteristic value, but it should be appreciated that thecoding parameter may be a motion compensating on/off threshold value, ora valid/invalid discrimination value. The scope of this invention is notintended to be limited by the kind of coding parameters that areemployed. In addition, the scope of this invention is not intended to belimited according to the kinds of produced information amount data.

In FIG. 19, A_(m) (N) is different in modes A, B and C becausereproducing the same extent of image quality produces much moreinformation in the intra-frame estimation mode (mode A) compared withthe value of A_(m) (N) in the inter-frame estimation mode (mode B ormode C). In order to keep the necessary image quality, much moreinformation must be produced in mode A or mode B, where the coded frameis used for the next estimation in contrast with the mode C where thecoded frame is not used for the next estimation. The renewed codingparameter Q is stored in the coding parameter renew part 140.

The relationship between the quantize characteristic value Q and theestimated error power P or the produced information amount differencevalue A-A_(m) (i.e., the difference between the produced informationamount A output from the memory 120 and the produced information amounttarget value A_(m)) is shown in FIG. 21. The larger the estimated errorpower P, the more the quantize characteristic value Q is raised as shownin FIG. 21. Further, the larger the produced information amountdifference value A-A_(m), the more the quantize characteristic value isincreased as shown in FIG. 21. As a result, the change of the producedinformation amount can be controlled so as to make the producedinformation amount close to the target value.

In this embodiment, although the coding process is carried out per blockunit and the coding control is executed per frame unit, the codingprocess may be performed per pixel unit or per block group unit thatincludes a plurality of blocks. In addition, the coding control can becarried out per block unit or per block group unit.

As shown in FIG. 21, although the quantize characteristic value isobtained, the basic quantize characteristic value Q may be determinedfrom the total produced information amount A_(mt) of the period T, andthen the offset of the quantize characteristic value Q may be obtained,depending on the produced information amount difference value A-A_(m)and the estimated error power P to renew the quantize characteristicvalue Q. Also, the offset can be added to the quantize characteristicvalue Q used in the preceding process unit to renew the quantizecharacteristic value Q.

In this embodiment, the valid/invalid block discrimination thresholdvalue (to be used for discrimination whether the block is valid orinvalid) may be used as the coding parameter. The present invention isnot restricted by the type or number of the coding parameters. Inaddition, intra-frame estimation of the block may be executed to carryout the coding. Although the both direction estimate is performed inevery block within the frames in the mode C, the intra-frame estimationor the inter-frame one direction estimation of the block may be used tocarry out the coding. Further, the coding modes are not restricted tothe three modes A, B and C. Other coding modes may be used, and thepresent invention is not restricted by the type or number of the codingmodes.

In this embodiment, although the difference absolute value sum is usedas the estimated error power 20, a difference square sum may also beused, and the present invention is not restricted by the type of theestimated error power. Moreover, as shown in FIG. 18, although oneperiod T includes 15 frames, the number of the frames included in oneperiod T may be varied, and the coding modes present within the periodmay be replaced with other coding modes. The present invention is notrestricted by the length of the period, the number of frames in oneperiod or the number of coding modes within the period. Lastly, althoughthe produced information amount 19 output from the transmission buffer18 is stored in the memory 120 in the illustrated embodiment, theproduced information amounts in one period may be further storedsimultaneously in the memory 120, and the processed value, such as atotal sum value of the produced information amounts in every unitprocess along with the produced information amounts in every unitprocess, may be stored.

The operation of the above-described embodiments may be summarized asfollows. The coder for coding a sequence of image signals is controlledby coding parameters that are sent to the coder. These coding parametersare updated in accordance with the steps shown in the flowchart of FIG.22. Specifically, the system provides an information amount of a currentimage signal and an average information amount for at least onepreviously encoded image signal (steps 224 and 226). The system thendetermines the extent of motion between the current information signaland the previously encoded signal based on the information amount andaverage information amount that has been provided (step 228). The codingparameters are then updated based on the determined extent of motion andcurrent coding parameters (step 230).

Although the present invention has been described in its preferredembodiments with reference to the accompanying drawings, it is readilyunderstood that the present invention is not restricted to the preferredembodiments and that various changes and modifications can be made bythose skilled in the art without departing from the spirit and scope ofthe present invention.

What is claimed is:
 1. In an image coding system having a coder forcoding a sequence of image signals, as dictated by a control signal thatsets coding parameters for the coder, a controller for controlling thecoder, comprising:a) a motion detector for detecting an extent of motionbetween an image signal and at least one previously encoded image signalbased on an information amount of said current image signal and anaverage information amount of said at least one previously encoded imagesignal; and b) a coding update means receiving the extent of motion fromsaid motion detector for generating the control signal to update thequantize condition in the coder based upon the extent of motion.
 2. Acontroller as recited in claim 1 wherein the coding update meanscomprises a coding parameter sets means that generates the controlsignal so that the coding produces a target information amount.
 3. In animage coding system having a coder for coding a sequence of imagesignals, as dictated by coding parameters for the coder, a method ofupdating the coding parameters, comprising the steps of:a) providing aninformation amount of a current image signal and an information amountfor at least one previously encoded image signal; b) determining anextent of motion between said current image signal and said at least onepreviously encoded image signal based on the provided currentinformation amount and previous information amount; and c) updating thecoding parameters by calculating an updated quantize condition based onthe determined extent of motion and a current quantize condition.
 4. Acoding control apparatus for coding a frame of image data according tomotion between image data of the frame and image data of at least oneprevious frame, comprising:a first memory for storing motioncharacteristics of image data of previous frames; a second memory forstoring a coding parameter to control coding of the image data of thepresent frame so that coded image data has a produced informationamount; a detector for detecting motion over a predetermined sequence offrames from the past motion characteristics held in the first memory;and coding parameter set means for updating a quantize condition in thecoding parameter to be output, based on the temporal change of themotion that is detected by the detector and based on a record of acurrent quantize condition in the coding parameter held in the secondmemory.
 5. An image coding process apparatus for carrying out a codingprocess of an input image signal, on a block by block basis, whereineach block includes a predetermined number of pixels, the input signaltemporally changing and including data of every pixel of a correspondingimage, comprising:a coder for coding each block using a quantizecondition according to the input image signal of the block to obtain aproduced information amount of coded data; a converter for convertingthe produced information amount of the coded data into a conversioncoded data amount, while maintaining a fixed relationship between thepredetermined quantize condition and the produced information amount; adetector for detecting motion characteristics of the input image signalbased on change in the conversion coded data amount of the input imagesignal and the at least one previously coded input image signal; andquantize update means for calculating the quantize condition andupdating the quantize condition in the coder according to the motioncharacteristics obtained by the detector.
 6. The apparatus of claim 5,wherein the quantize condition includes a quantize characteristic valuerepresenting a size of each step when the data are quantized.
 7. Theapparatus of claim 5, further including means for detecting the producedinformation amount of the coded data, and control means for renewing acoding parameter according to the detected produced information amount.8. The operation of claim 7, wherein said control means renews thecoding parameter based on inference.
 9. The apparatus of claim 7,wherein said control means renews the coding parameter according to theproduced information amount and quantize condition.
 10. The apparatus ofclaim 7, wherein operation of said control means in renewing the codingparameter is controlled by calculating the produced information amountof said in terms of the conversion coded data amount, while maintaininga fixed relationship between the predetermined quantize condition andthe produced information amount.